Method and system for processing an image of a biological specimen

ABSTRACT

Components, e.g., background, cytoplasm, nucleus and nucleolus, of a biological specimen are identified using multi-wavelength analysis. Specimen components, such as nucleoli, are selected, and a determination is made whether cells having nucleoli are cancer cells or regular repair cells based on one or more physical characteristics of the identified component. The physical characteristics can be one or more of a shape, size, texture and gray value.

RELATED APPLICATIONS DATA

The present application is a continuation of pending U.S. patentapplication Ser. No. 11/957,084, filed Dec. 14, 2007, which claims thebenefit under 35 U.S.C. §119 to U.S. provisional patent application Ser.No. 60/870,838, filed Dec. 19, 2006. The foregoing applications arehereby incorporated by reference into the present application in theirentirety.

FIELD OF INVENTION

The invention relates to imaging and analysis of biological specimens,and more particularly, to reducing the number of non-cancerous repaircells that are selected by an imaging system for subsequent review.

BACKGROUND

Medical professionals and cytotechnologists frequently review biologicalspecimens affixed to a specimen carrier, such as a slide, to analyzewhether a person from whom the specimen was obtained has or may have aparticular medical condition. For example, it is well known to examine acytological specimen in order to detect the presence of malignant orpre-malignant cells as part of a Papanicolaou (Pap) smear test. Tofacilitate this review process, automated systems have been employed toperform a pre-screening of the specimen slides in order to focus thecytotechnologist's attention on the most (or at least more) pertinentcells or groups of cells in the respective specimen, while discardingless relevant cells from further review. One such automated imagingsystem is the Thinprep Imaging System, available from Cytyc Corporation,250 Campus Drive, Marlborough, Mass. 01752 (www.cytyc.com).

FIG. 1 generally illustrates a known imaging system 10 that includes aprocessor, computer or controller 11, an optical stack 12 and a robotfor feeding and removing specimen slides 14 to and from the opticalstack 12. An optical stack 12 includes a motion control board computeror controller 20, a stage 21, a light source 22, a lens 23 and a camera24. Images generated by the optical stack 12 are provided to thecomputer 11 for analysis. The robot 13 takes a slide 14 from a cassette30 and places the slide 14 on the stage 21. The computer 11 controls theMCB computer 20 so that the MCB computer 20 moves the stage 21 tolocation the slide 14 under the camera 24 and the lens 23. The lightsource 22 is activated, and an image of a portion of the specimen on theslide 14 is acquired by the camera 24 and provided to the computer 11.The computer 11 instructs the MCB computer 20 to move the stage 21 andthe slide 14 thereon a very short distance from a first location to asecond location. An image of the next portion of the specimen on theslide 14 at the second location is acquired by the camera 24 andprovided to the computer 11.

The stage 21 is moved to a different location after an image is taken ofdifferent portions of the specimen on the slide 14. A first portion ofthe specimen is imaged when the stage 21 is at a first stage location.The stage 21 is moved to a second location, and an image of a secondportion of the specimen is acquired at the second location. The stage 21is moved to a third location, and an image of the third portion of thespecimen is acquired, and so on for each portion of the specimen untilthe entire specimen is imaged. In known imaging systems, the stage 21can be moved about 2,400 times to acquire 2,400 images of 2,400different portions of a specimen. The robot 13 then removes the imagedslide 14 from the stage 21 and places another slide 14 from the cassette30 onto the stage 21 for imaging as described above.

After images of the specimen are acquired, the images are processed toidentify or rank cells and cell clusters that are of diagnosticinterest. In some systems, this includes identifying those cells thatmost likely have attributes consistent with malignant or pre-malignantcells and their locations (x-y coordinates) on the slide. For example,the processor 11 may select about 20 fields of view, e.g., 22 fields ofview, which include x-y coordinates identifying the locations of cellsand cell clusters that were selected by the processor 11. This field ofview or coordinate information is provided to the microscope (not shownin FIG. 1), which steps through the identified x-y coordinates, placingthe cells or clusters of cells within the field of view of thetechnician. While current imaging systems and methods for selectingportions of images for further review have been used effectively in thepast, they can be improved.

For example, referring to FIG. 2, if the cells are consistent withpre-malignant cells or malignant or cancerous cells 42 (generallyidentified by “C”), then the selected fields of view 40 ideally identifythese cells or regions so that the cytotechnologist is directed to thosecells or clusters during review. Although FIG. 2 illustrates each fieldof view 40 having cancerous cells, it should be understood that somefields of view have regular cells, whereas other fields of view havenon-cancerous cells, but the processor 11 is configured to identifycells or regions 42 that are most consistent with pre-malignant andmalignant or cancerous cells.

Referring to FIG. 3, in some cases, however, there may be normal,non-cancerous cells, e.g., repair cells 44 (identified by “R”), whichare normally dividing cells that are generated to replace or repairdamaged tissue. These repair cells 44 may appear similar topre-malignant or malignant cells 42 that would otherwise be selected bythe processor 11 since repair cells 44 and cancerous cells 42 bothinclude dividing nucleus components. Consequently, normal repair cells44 may result in “false alarms” in that they may be ranked higher thanother malignant or pre-malignant cells 42 that would otherwise beselected by a processor 11 if the false alarm cells were not selected.Thus, when repair cells 44 are analyzed, the processor 11 may selectfields of view that include higher ranking non-cancerous cells insteadof other fields of view that include more relevant cells (possiblycancerous cells). The fields of view that should have been selected aregenerally illustrated by dotted lines in FIG. 3. Thus, thecytotechnologist may not be presented with the most relevant fields ofview, possibly resulting in a less accurate diagnosis.

SUMMARY OF THE INVENTION

In one embodiment, a method of processing an image of a biologicalspecimen having a plurality of cells includes identifying components ofthe biological specimen based on spectral characteristics of thecomponents, selecting an identified component, and determining whethercells having the selected component have a likelihood of being cancerousor non-cancerous based on one or more physical characteristics of theidentified component. By way of non-limiting examples, the identifiedcomponents of the specimen may include one or more of background,cytoplasm and nuclear (e.g., nucleolus) components of the specimen. Byway of further non-limiting examples, the components of the specimen maybe identified using one or more light wavelengths in a range from about400 nm to about 720 nm. The components may be identified by classifyinga pixel of the image as a certain component of the biological specimenbased on spectral characteristics of the pixel. The method mayoptionally further include determining whether cells corresponding tothe selected components are cancer cells or normal repair cells. Thedetermination of whether cells having the selected component have alikelihood of being cancerous or non-cancerous may be based on one ormore of a shape, a texture or a size of the identified component.Additionally or alternatively, the determination of whether cells havingthe selected component have a likelihood of being cancerous ornon-cancerous may be based on a transmittance or gray value of theidentified component or of an image pixel of the identified component.

In another embodiment, a method of processing an image of a biologicalspecimen having a plurality of cells includes identifying components ofthe biological specimen based on spectral characteristics of thecomponents, the identified components including nucleus components. Themethod further includes selecting regions of the image having nucleuscomponents having one or more nucleolus components, and determiningwhether cells in the selected regions are cancer cells or repair cellsbased on one or more physical characteristics of the nucleoluscomponents. Again, the determination of whether cells in the selectedregions are cancer cells or repair cells may be based on one or more ofa shape, a size, a texture and a transmittance of the nucleoluscomponents.

In still another embodiment, a method of processing an image of abiological specimen having a plurality of cells includes selecting afield of view from a plurality of fields of view of the image, selectinga field of view, classifying pixels of selected field of view asbackground, cytoplasm, nucleus or nucleolus components based on spectralcharacteristics of the pixels, selecting pixel regions of the selectedfield of view that correspond to nucleolus components, and determiningwhether cells in the selected pixel regions are cancer cells or repaircells based on one or more physical characteristics of the nucleoluscomponents. The physical components may include one or more of a shape,a size, a texture and a transmittance of the nucleolus components.

In yet another embodiment, a system for processing an image of abiological specimen having a plurality of cells includes, an imagerhaving a processor configured to identify components of the biologicalspecimen based on spectral characteristics of the components, select anidentified component, and determine whether cells in the image havingthe selected component are cancerous or non-cancerous based on one ormore physical characteristics of the identified component. The processormay identify components of the specimen based on identifying one or moreof background, cytoplasm and nuclear (e.g., nucleolus) components of thespecimen. The components of the specimen may be identified using one ormore light wavelengths in a range from about 400 nm to about 720 nm,wherein the system preferably includes a light source or light sourcesof the one or more light wavelengths, and wherein the processoridentifies components of the specimen based on classifying a pixel ofthe image as a certain component of the biological specimen based onspectral characteristics of the pixel. The processor may be furtherconfigured to determine whether cells corresponding to the selectedcomponents are cancer cells or normal repair cells based on one or moreof a shape, a texture, a size and a transmittance of the identifiedcomponent.

BRIEF DESCRIPTION OF THE DRAWINGS

Referring now to the drawings in which like reference numbers representcorresponding parts throughout and in which:

FIG. 1 illustrates a known specimen slide imaging system;

FIG. 2 generally illustrates fields of view including cells identifiedas having attributes consistent with malignant or pre-malignant cells;

FIG. 3 generally illustrates fields of view including normally dividingcells and that are selected to the exclusion of other fields of viewthat may include cells having attributes consistent with malignant orpre-malignant cells;

FIG. 4 is a flow chart of a method of processing images of a biologicalspecimen involving analysis of spectral and physical characteristicsaccording to one embodiment;

FIG. 5 is a more detailed flow chart of a method of processing images ofa biological specimen involving analysis of spectral and physicalcharacteristics according to one embodiment;

FIG. 6 generally illustrates spectral analysis of a portion or region ofa biological specimen;

FIG. 7 generally illustrates identifying components of a portion orregion of a biological specimen as a result of spectral analysis;

FIG. 8 is an image obtained using spectral analysis and showingcomponents of a normal repair cell cluster;

FIG. 9 is a gray scale image showing components of a normal repair cellcluster in less detail than the image shown in FIG. 7;

FIG. 10 generally illustrates segmentation or selection of nucleuscomponents;

FIG. 11 generally illustrates segmentation or selection of nucleoluscomponents;

FIG. 12 is an image illustrating different physical properties of cancercells compared to normally dividing, non-cancerous cells;

FIG. 13 is a flow chart a method of processing an image of a biologicalspecimen involving analysis of spectral and physical characteristicsaccording to another embodiment;

FIG. 14 generally illustrates identification and rejection of repaircells so that a field of view including repair cells is not selected forfurther review and analysis;

FIG. 15 is a flow chart showing a method of processing an image of abiological specimen based on transmission characteristics according toanother embodiment;

FIG. 16 generally illustrates determining gray values at differentwavelengths of light transmitted through a nucleus of a cancer cell;

FIG. 17 generally illustrates determining gray values at differentwavelengths of light transmitted through a nucleus of a regularlydividing repair cell;

FIG. 18 illustrates transmission profiles of gray values of cancer andrepair cells;

FIG. 19 is a flow chart of a method of processing an image of abiological specimen based on transmission characteristics according toanother embodiment;

FIG. 20 is a chart comparing light intensity and optical density ofpixels digitized by using different wavelength of light to transmissionprofiles of cancer and repair cells according to one embodiment;

FIG. 21 illustrates transmission and optical density profiles of cancerand repair cells; and

FIG. 22 illustrates a diagram of a computer hardware system that can beused to perform various functions described herein in accordance withsome embodiments.

DETAILED DESCRIPTION OF ILLUSTRATED EMBODIMENTS

Referring to FIG. 4, according to one embodiment of the invention, amethod 400 of processing an image of a biological specimen, e.g. acytological specimen, includes identifying components of the specimenusing spectral analysis in step 405. After components are identified, instep 410, one of the components is selected. For example, the selectedcomponent may be a nucleolus of a nucleus. Then, in step 415, adetermination is made whether cells that include the selected componentare cancerous or non-cancerous based on one or more physicalcharacteristics of the component.

Referring to FIG. 5, a method 500 of processing an image of a biologicalspecimen includes receiving a pixel that is to be imaged in step 505. Instep 510, spectral analysis is performed to determine whether a pixel isbackground, cytoplasm, nucleus or nucleolus. For purposes of explanationand illustration, reference is made to nucleus and nucleolus componentssince background and cytoplasm components can be disregarded forpurposes of this analysis.

Continuing with step 515, a determination is made whether the pixelbelongs to a nucleolus. If it is determined that the pixel belongs to anucleolus, then the pixel is labeled as such in step 520. Otherwise, ifit is determined that the pixel does not belong to a nucleolus, then thepixel belongs to a nucleus and is labeled as such in step 525.

In step 530, a determination is made whether any additional pixels ofthe image should be classified. If so, then additional pixels areclassified beginning with step 505 until the entire image is processed.Otherwise, after all of the pixels classified, e.g., as either part of anucleolus or a nucleus, then in step 535, a determination is made as towhich nucleolus components or nucleoli belong to which nucleus.

In step 540, physical features of nucleolus components can be analyzed,e.g., the shape, size and texture of the nucleolus. Additionally, ifnecessary, in step 545, a statistical analysis of nucleus components canbe performed. Step 550 may involve, for example, the number of nucleoluscomponents within a nucleus, whether the size of nucleolus componentsvaries and if so, by how much, the mean size of nucleoli, standarddeviation of sizes of nucleoli, the largest nucleoli and the smallestnucleoli, probabilities, weighted probabilities and other suitablestatistical functions.

Following analysis of physical characteristics (step 545) andstatistical analysis, in step 555 a determination is made whether agiven nucleus is cancerous or non-cancerous (e.g. a repair cell). Step555 can be performed using, for example, linear discriminant analysis,Bayesian Network, hierarchical trees. This analysis can determine whichphysical characteristic and/or statistical variables are the bestpredictors to classify a nucleus as a cancer cell or a repair cell.

For example, in some embodiments, training data can be acquired todetermine size, shape and texture data for cancer and repair cellsamples. After the training data is acquired, a covariance matrix andlinear discriminant analysis can be computed to indicate how significanta physical and/or statistical feature is to identify a nucleus as partof a cancer cell or a normal repair cell. The linear discriminantfunction is used to compute a predictor or value that allows forclassification of new physical characteristic data from cells that arebeing examined/reviewed.

Following acquisition and processing of the preliminary training dataand derivation of the linear discriminant function, actual physicalcharacteristic data (step 545) of incoming pixels (step 505) can beanalyzed to determine, in step 555, whether a given nucleus is cancerousor non-cancerous based on the previously acquired training data. Forexample, physical characteristic data that is acquired during step 545can be input into the previously derived linear discriminant function.Fitting the data into a linear discriminant function that represents therepair class provides a Repair Score (RS). By fitting the data into alinear discriminant function that represents the cancer class provides aCancer Score (CS). If CS is greater than RS, then it can be determinedthat the nucleus belongs to the cancer class and vice versa. Thisanalysis also involves computing the Mahalanobis Distance anddetermining the shortest distance of the features from their group meansto indicate whether a given nucleus is more likely part of a repair cellor a cancer cell. Persons skilled in the art will appreciate thatvarious other functions and analyses can be utilized to determinewhether a given nucleus is part of a cancer or repair cell. Thus, theexemplary functions described above are provided for purposes ofexplanation in a non-limiting manner.

In step 555, if it is determined that the nucleus belongs to a cancercell, the pixel or pixels comprising the nucleus can be accepted forfurther review. Otherwise, in step 560, the repair cell nucleus can berejected if it is determined that the nucleus belongs to a normallydividing repair cell so that the normal cell is not selected for furtherreview. Further aspects of spectral analysis to identify cellularcomponents and analysis of physical nucleoli characteristics and/ornuclear statistical characteristics to determine whether a given nucleusis part of a cancer or repair cell are explained with reference to FIGS.6-14.

Referring to FIG. 6, spectral analysis involves exposing a portion 60 ofthe image to different wavelengths 62 of light (generally represented byλ1-λn). According to one embodiment, the range of wavelengths usedduring spectral analysis 62 of the specimen can be from about 400 nm toabout 720 nm, e.g., in 10 nm increments. Spectral analysis can involve,for example, about three to about 30 wavelengths.

Persons skilled in the art will appreciate that different ranges ofwavelengths, numbers of wavelengths and wavelength increments can beutilized. The portion 60 can also be various shapes and sizes andinclude one or more cells. For example, the portion 60 can be a selectedfield of view. For purposes of illustration and explanation, theselected portion 60 is shown as having a square shape and comprised ofone or more pixels 64.

Referring to FIG. 7, different components of the specimen portion 60 areidentified or classified as a result of multi-wavelength spectralanalysis 62. In the illustrated embodiment, pixels 70 of the image ofthe portion 60 (pixels 64 of the image) are classified as background 72,cytoplasm 74 and a nucleus 76, which includes smaller nucleolus 78components. A nucleolus 78 is a generally round or oval and dense bodythat includes DNA and RNA. Embodiments advantageously utilize multiplewavelengths 62 to identify the various specimen components and to allownucleoli 78 to become more visible and distinguishable compared toimages produced by known imaging systems. The results and advantages ofembodiments that utilize spectral analysis 62 are further illustrated bycomparing FIGS. 8 and 9.

Referring to FIG. 8, multi-wavelength spectral analysis 62 allowschromatin details and nucleoli 78 to be clearly visible within thenucleus 76. For purposes of comparison, FIG. 9 is a gray scale imagewhich shows nucleus 76, nucleolus 78 and cytoplasm 74 components, butthese components are clearly less distinctive and less visible comparedto the nucleus 76, nucleolus 78 and cytoplasm 74 components shown inFIG. 8 generated with multiple wavelengths 62. Thus, embodiments usemulti-wavelength spectral analysis 62 to advantageously identifydifferent components, including nucleoli 78 components, with enhancedvisibility and detail compared to gray scale images.

Referring to FIGS. 10 and 11, pixels 64 corresponding to selectedcomponents of the specimen can be segmented 100 or separated from othercomponents after pixels 64 are identified or classified using spectralanalysis 62. For example, referring to FIG. 10, nucleus 76 componentscan be segmented 100 based on the components identified as shown in FIG.8. FIG. 10 shows nucleus 76 components being enclosed by a light grayborder. Further, referring to FIG. 9, nucleolus 78 components can besegmented 110 based on the identified components shown in FIG. 8. FIG.11 shows nucleoli 78 components that are enclosed by a light grayborder.

Physical characteristics of the nuclear components can be analyzed todetermine whether the cells of the pixels 64 are cancerous ornon-cancerous after the nuclear component pixels 64 are identified orsegmented. For example, according to one embodiment, nucleoli 78components are identified, and one or more physical characteristics ofthe nucleoli 78 are analyzed to determine whether the correspondingcells are non-cancerous repair cells 44 or cancerous cells 42.

According to one embodiment, a physical characteristic of the nucleoli78 that is used to identify repair cells 44 and distinguish repair cells44 from cancerous cells 42 is the shape of the nucleoli 78. According toone embodiment, a physical characteristic of the nucleoli 78 within thenucleus 76 that is used to identify repair cells 44 and distinguishrepair cells 44 from cancerous cells 42 is the variation of size of thenucleoli 78 within the nucleus 76. According to another embodiment, aphysical characteristic of the nucleoli 78 within the nucleus 76 that isused to identify repair cells 44 and distinguish repair cells 44 fromcancerous cells 42 is the texture of the nucleoli 78 and/or nucleus 76.In a further embodiment, a physical characteristic of the nucleoli 78and/or nucleus 76 that is used to identify repair cells 44 anddistinguish repair cells 44 from cancerous cells 42 is the variation ofgray value of the pixel within the nucleoli 78. Persons skilled in theart will appreciate that other physical characteristics can be utilized,and that shape, size, texture and gray value are exemplarycharacteristics that can be used with embodiments.

In addition to considering physical characteristics individually, twophysical characteristics, three characteristics, or all of the physicalcharacteristics can be considered. For example, a determination whethera nucleus 76 is part of a cancerous cell 42 can be based on acombination of shape and size, shape and texture, shape andtransmittance, size and texture, size and transmittance and shape andtransmittance. Further, a determination whether a nucleus 76 is part ofa cancerous cell 42 or a non-cancerous repair cell 44 can be based onthree different physical characteristics, e.g., shape, size and texture,shape, size and transmittance, size, texture and transmittance, textureshape and size, etc. Further, all four of the exemplary physicalcharacteristics can be utilized.

The use of physical characteristics (and nuclear statistical analysis ifnecessary) to distinguish repair cells 44 and cancer cells 42 is shownwith reference to FIGS. 8 and 12. As shown in FIGS. 8 and 12, nucleoli78 within cancer cells 42 have an irregular shape or border whereasnucleoli 78 within repair cells 44 have consistent or smooth shapedborders. As a further example, nucleoli 78 within cancer cells 42 arehighly textured, whereas nucleoli 78 within repair cells 44 are not.Additionally, the sizes of nucleoli 78 components in cancer cells 42 canvary, whereas nucleoli 78 components of repair cells 44 are smaller andmore consistent in size.

Spectral analysis 62 and physical characteristic analysis can be appliedto the entire image or portions or regions of the image, which can bevarious shapes and sizes. According to one embodiment, referring to FIG.13, a method 1300 of processing an image of a specimen includesanalyzing individual fields of view. The method 1300 includes selectinga plurality of fields of view in step 1305. According to one embodiment,each field of view is processed individually so that in step 1310, oneof the fields of views is selected. In step 1315, pixels of the selectedfield of view are classified as a component, e.g., background,cytoplasm, nucleus and nucleoli components. In step 1320, pixel regionsthat correspond to components, such as nucleolus components, that areused to differentiate repair and cancer cells, are selected, and in step1325, a determination is made whether a nucleus corresponding to theselected pixels is a repair cell or a cancer cell.

In a further alternative embodiment, statistical analysis of a nucleus76 can be performed independently or in combination with analysis ofphysical characteristics of nucleoli 78 to determine whether a givennucleus 76 is part of a cancer cell 42 or a repair cell 44. Statisticalanalysis of nucleus 76 may involve determining the number of nucleoli 78within a nucleus 76. Cancer cells 42 and repair cells 44 can bedifferentiated based on a nucleus 76 of a cancer cell 42 having morethan one nucleolus 78, whereas a nucleus 76 of a repair cell 44typically has one or two distinguishable nucleoli 78. Statisticalanalysis of nucleus 76 may also involve determining the degree to whichthe size of nucleoli 78 vary, the mean size of nucleoli 78, standarddeviation of the size of nucleoli 78, the largest nucleoli 78, thesmallest nucleoli 78, the darkest nucleoli 78, the lightest nucleoli 78and posterior probability of whether this nucleolus belongs to acancerous nucleus.

Referring to FIG. 14, identified repair cells 44 can be rejected andeliminated from further consideration so that they do not outrank andexclude other possibly more relevant cells from further review by acytotechnologist after repair cells 44 are identified using spectralanalysis 52 and analysis of suitable physical characteristics (andstatistical analysis of nucleus 76 as necessary). In the illustratedexample, FIG. 13 illustrates 11 fields of view 40. Two fields of view 40include repair cells 44, which were erroneously identified as beingpre-malignant or malignant cells or cell clusters as a result of havingdividing nuclear components similar to cancer cells 42. Embodimentsaddress these shortcomings by using spectral analysis 62 and analysis ofphysical characteristics to reliably identify repair cells 44 and rejectthe identified repair cells 44, as indicated by an “X” through therepair cell “R” fields of view 40 that were initially identified. Newfields of view 40 can then be selected to replace the rejected fields ofview 40. Thus, embodiments advantageously maximize the number of fieldsof view 40 that include cells that most likely have attributesconsistent with malignant or pre-malignant cells.

In the embodiment shown in FIG. 14, the imager has selected an initialset of fields of view 40. Fields of view including repair cells can berejected and replaced with other fields of view 40. Thus, rejecting andreplacing a repair cell field of view occurs after the imager hasalready selected a set of fields of view. In an alternative embodiment,physical characteristic analyses can be performed while images are beingacquired or before an initial set of fields of view is generated so thatthe fields of view that are eventually generated already incorporate theresults of spectral and physical characteristic analysis. Thus, in theseembodiments, it may not be necessary to subsequently identify fields ofview having repair cells 44 and replace the identified fields of viewwith other fields of view at a later time since the repair cells 44 wereidentified and rejected during image acquisition or prior to generationof the fields of view.

FIGS. 15-21 illustrate other embodiments directed to identifying repaircells 44 by comparing transmittance of light through nucleus componentsof selected cells or pixels and transmittance of light through nucleuscomponents of other cells in order to determine whether the cells underexamination are repair cells 44 or cancer cells 42. Thus, whileembodiments described with reference to FIGS. 4-14 can be implemented byexamining transmittance of light through sub-nuclear nucleolus 78components, alternative embodiments can be implemented by examiningtransmittance of light through nucleus 76 components.

Referring to FIG. 15, a method 1500 for classifying a nucleus as acancer cell or repair cell according to an alternative embodimentincludes transmitting light at multiple wavelengths through a region,e.g., a group of pixels, of nucleus components of cancer and repaircells in step 1505. In step 1510, gray values of the nucleus regions aremeasured at the various wavelengths in order to generate a set oftraining or reference data that includes gray values of nucleuscomponents of cancer cells and gray values of nucleus components ofrepair cells. Subsequent gray value measurements of nucleus componentsof cells that are being reviewed can be compared against the trainingdata.

After training data is acquired, in step 1515, light at multiplewavelengths is transmitted through a nucleus region of a cell that isbeing reviewed or examined. In step 1520, the gray values of the nucleusregion being reviewed are determined. In step 1525, the measured grayvalues are compared to the previously determined training data/grayvalues.

Then, in step 1530, a determination is made whether the nucleus regionbeing reviewed is part of cancerous or non-cancerous cell based on thecomparison of the measured gray values and the training data. Personsskilled in the art will appreciate that although embodiments aredescribed with reference to transmittance and gray values, other methodscan also used. Thus, references to transmission characteristics areprovided for purposes of illustration and explanation since absorptioncharacteristics and profiles can also be utilized.

Referring to FIGS. 16 and 17, according to one embodiment, transmittanceof light through the cancer cells 42 and repair cells 44 is determinedby spectral analysis 62 of nucleus 76 components. For this purpose,nucleus 76 components can be segmented from other portions of an imageor a field of view, e.g., by using DVC segmentation or othersegmentation methods. As shown in FIG. 16, light at a first wavelengthis passed through a nucleus 76 of a cancer cell 42, light at a secondwavelength is passed through a nucleus 76 of a cancer cell 42, and so onfor as many wavelengths as necessary in order to determine transmissioncharacteristics of cancer cells 42. Similarly, referring to FIG. 17,light at a first wavelength is passed through a nucleus 76 of a repaircell 44, light at a second wavelength is passed through a nucleus 76 ofa repair cell 44, and so on for as many wavelengths as necessary inorder to determine transmission characteristics of repair cells 44.

According to one embodiment, light at about three to about 30 differentwavelengths ranging from about 400 nm to about 720 nm is utilized todetermine transmission characteristics of cancer cells 42 and repaircells 44. Other wavelengths and other numbers of wavelengths can beutilized. In one test, a collection of about 2,000 spectral data pointsrepresenting segmented repair cells 44 was use to determine transmissioncharacteristics of repair cells 44 using wavelengths ranging from 400 nmto 720 nm. A collection of about 3,000 spectral data points representingsegmented cancer cells 42 was used to determine transmissioncharacteristics of cancer cells 42 using these same wavelengths.Referring to FIG. 18, a transmission profile 180 or collection of grayvalues of cancer cells 42 at various wavelengths and a transmissionprofile 182 of gray values of repair cells 44 at various wavelengths canbe generated based on the transmission of light at various wavelengthsthrough cancer cells 42 and repair cells 44 as shown in FIGS. 16 and 17.

Referring to FIGS. 19 and 20, in one embodiment, a method 1900 of usingtransmittance comparisons to distinguish cancer cells 42 and repaircells 44 includes determining a transmission profile 180 of cancer cellsby quantifying the transmitted light through nuclear regions of thecancer cells at various wavelengths and measuring gray values of thenuclear regions at the various wavelengths in step 1905. In step 1910, atransmission profile 182 of repair cells is determined by quantifyingthe transmitted light through nuclear regions of repair cells at variousnumbers of wavelengths and measuring the gray values at the variouswavelengths, e.g., the same wavelengths used in step 1905 to allowdirect comparisons. In step 1915, nuclear regions of an image beingprocessed can be segmented or identified. In step 1920, a nuclear regionis selected, and in step 1925, light at different wavelengths istransmitted through the selected nuclear region. For example, thewavelengths in step 1925 can be the wavelengths that were used togenerate the transmission profiles 180 and 182.

In step 1930, the gray value of the selected nuclear region at eachwavelength is measured. The collection of measured gray values isrepresented by measured gray value data 200 in FIG. 20. In step 1935,the measured gray values 200 are compared to the gray values of thecancer cell transmission profile 180. Further, in step 1940, themeasured gray values 200 are compared to the gray values of the repaircell transmission profile. In step 1945, a determination is made whethera measured gray value at a given wavelength matches or is more similaror closer to a gray value of cancer cell profile or a repair cellprofile at that wavelength. The results of the determinations in step1945 are shown as data 202. One manner in which step 1945 can be carriedout is described below.

Each pixel within the nucleus is represented by a feature vector. Eachfeature vector is composed of n different gray values and n differentoptical density values, where “n” is equal to the number of differentwavelength of light being used. Optical density is the logtransformation of the gray values. For example, if a nucleus has 150pixels, then each feature vector has 2×n feature values and each nucleusis represented by 150 feature vectors. These feature vectors are thencompared to the training data. The linear discriminant functions arecomputed from the training data. The training data is composed of alarge number of feature vectors that are pre-calculated from thecancerous nuclei and repair nuclei.

A Posterior Probability of being Cancer (PPC) is calculated for eachfeature vector during the linear discriminant analysis. A PosteriorProbability of being Repair (PPR) is calculated for each feature vectorduring the linear discriminant analysis. The average PPC and the averagePPR of all the feature vectors that belong to the same nucleus arecomputed. If the average PPC is greater than the average PPR, then thisnucleus is classified as cancer. The cell that contains this nucleus isa cancer cell.

Similar to the embodiments shown in FIGS. 4-14, embodiments shown inFIGS. 14-20 can be implemented before or after an initial set of fieldsof view is generated by an imager. Thus, embodiments are described withreference to an imager that has selected an initial set of fields ofview 40. If certain fields of view include repair cells, ten they arerejected and replaced with other fields of review. Thus, the repair cellrejection and replacement occurs after the imager has already selected aset of fields of view. In an alternative embodiment, spectraltransmission characteristics can be considered while images are beingacquired or before an initial set of fields of view is generated so thatthe fields of view that are eventually generated already incorporate theresults of transmittance comparisons. Thus, in these embodiments, it maynot be necessary to subsequently identify fields of view having repaircells and replace the identified fields of view with other fields ofview at a later time since the repair cells were identified and rejectedduring image acquisition or prior to generation of the fields of view.

In a further alternative embodiment, the method described with referenceto FIGS. 14-21 based on measured gray values and transmission profilescan be combined with aspects of the method described with reference toFIGS. 4-14. For example, in order to supplement and/or confirm theconclusions reached with transmittance testing (FIG. 14-20), alternativeembodiments also analyze physical characteristics of nuclear componentsto provide further information to reliably identify cells or cellclusters as repair or cancer cells. Thus, for example, physicalcharacteristics of shape, size and texture can be used in combinationwith transmittance testing as necessary.

In yet another alternative embodiment, the method described withreference to FIG. 21 based on measured gray values and optical densitiesand transmission and optical density profiles can be combined withaspects of the method described with reference to FIGS. 4-14.

In the embodiment illustrated and described with reference to FIGS.14-21, transmittance measurements and comparisons are performed withoutreference to physical characteristics of nuclear components. In analternative embodiment, embodiments described with reference to FIGS.14-21 can be combined with embodiments described with reference to FIGS.4-14. Thus, in addition to transmittance measurements and comparisons,physical characteristics of nucleus can be considered to provideadditional information for distinguishing repair cells and cancerouscells.

In various embodiments of the invention, a substantial number ofartifacts or false alarm cells can be reliably identified and rejectedso that they are not selected for further review and analysis. Variousembodiments can also enable new types of imagers involving slide sortingand diagnosis, and can be implemented within an imager or using aseparate system. Thus, for example, a processor of the imager, such asprocessor 11 of imager 10 shown in FIG. 1, can be configured orprogrammed to execute embodiments. Alternatively, a separate processor,e.g., a separate computer that is not part of the imager, can be used toexecute embodiments. Embodiments of the invention can be implemented invarious imaging systems, including but not limited to the ThinprepImaging System manufactured and distributed by Cytyc Corporation.

FIG. 22 illustrates an exemplary known computer architecture that can beused to implement embodiments. One or more instructions can be importedinto a computer to enable the computer to perform any of the functionsdescribed herein. FIG. 22 is a block diagram that illustrates anexemplary computer system 220 that includes a bus 222 or othercommunication mechanism for communicating information, and a processor224 coupled with bus 222 for processing information. Computer system 220also includes a main memory 226, such as a random access memory (RAM) orother dynamic storage device, coupled to bus 222 for storing informationand instructions to be executed by processor 224. Main memory 226 alsomay be used for storing temporary variables or other intermediateinformation during execution of instructions to be executed by processor224. Computer system 220 may further include a read only memory (ROM)228 or other static storage device(s) coupled to bus 222 for storingstatic information and instructions for processor 224. A data storagedevice 230, such as a magnetic disk or optical disk, is provided andcoupled to bus 222 for storing information and instructions.

Computer system 220 may be coupled via bus 222 to a display 232, such asa cathode ray tube (CRT), for displaying information to a user. An inputdevice 234, including alphanumeric and other keys, is coupled to bus 222for communicating information and command selections to processor 224.Another type of user input device is cursor control 236, such as amouse, a trackball, cursor direction keys, or the like, forcommunicating direction information and command selections to processor224 and for controlling cursor movement on display 232. This inputdevice typically has two degrees of freedom in two axes, a first axis(e.g., x) and a second axis (e.g., y), that allows the device to specifypositions in a plane.

Embodiments of the invention described herein are related to the use ofcomputer system 220 for processing electronic data, and/or controllingan operation of the slide preparation machine 12. According to someembodiments, such use may be provided by computer system 220 in responseto processor 224 executing one or more sequences of one or moreinstructions contained in the main memory 206. Such instructions may beread into main memory 226 from another computer-readable medium, such asstorage device 230. Execution of the sequences of instructions containedin main memory 226 causes processor 224 to perform the process stepsdescribed herein. One or more processors in a multi-processingarrangement may also be employed to execute the sequences ofinstructions contained in main memory 226. In alternative embodiments,hard-wired circuitry may be used in place of or in combination withsoftware instructions to implement various operations/functionsdescribed herein. Thus, embodiments of the invention are not limited toany specific combination of hardware circuitry and software.

The term “computer-readable medium” as used herein refers to any mediumthat participates in providing instructions to processor 224 forexecution. Such a medium may take many forms, including but not limitedto, non-volatile media, volatile media, and transmission media.Non-volatile media includes, for example, optical or magnetic disks,such as storage device 230. Volatile media includes dynamic memory, suchas main memory 226. Transmission media includes coaxial cables, copperwire and fiber optics, including the wires that comprise bus 222.Transmission media can also take the form of acoustic or light waves,such as those generated during radio wave and infrared datacommunications.

Common forms of computer-readable media include, for example, a floppydisk, a flexible disk, hard disk, magnetic tape, or any other magneticmedium, a CD-ROM, any other optical medium, punch cards, paper tape, anyother physical medium with patterns of holes, a RAM, a PROM, and EPROM,a FLASH-EPROM, any other memory chip or cartridge, a carrier wave asdescribed hereinafter, or any other medium from which a computer canread.

Various forms of computer-readable media may be involved in carrying oneor more sequences of one or more instructions to processor 224 forexecution. For example, the instructions may initially be carried on amagnetic disk of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem local to computer system 220 canreceive the data on the telephone line and use an infrared transmitterto convert the data to an infrared signal. An infrared detector coupledto bus 222 can receive the data carried in the infrared signal and placethe data on bus 222. Bus 222 carries the data to main memory 226, fromwhich processor 224 retrieves and executes the instructions. Theinstructions received by main memory 226 may optionally be stored onstorage device 230 either before or after execution by processor 224.

Computer system 220 also includes a communication interface 218 coupledto bus 222. Communication interface 238 provides a two-way datacommunication coupling to a network link 240 that is connected to alocal network 242. For example, communication interface 238 may be anintegrated services digital network (ISDN) card or a modem to provide adata communication connection to a corresponding type of telephone line.As another example, communication interface 238 may be a local areanetwork (LAN) card to provide a data communication connection to acompatible LAN. Wireless links may also be implemented. In any suchimplementation, communication interface 218 sends and receiveselectrical, electromagnetic or optical signals that carry data streamsrepresenting various types of information.

Network link 240 typically provides data communication through one ormore networks to other devices. For example, network link 240 mayprovide a connection through local network 242 to a host computer 244.Network link 240 may also transmits data between an equipment 246 andcommunication interface 238. The data streams transported over thenetwork link 240 can comprise electrical, electromagnetic or opticalsignals. The signals through the various networks and the signals onnetwork link 240 and through communication interface 238, which carrydata to and from computer system 220, are exemplary forms of carrierwaves transporting the information. Computer system 220 can sendmessages and receive data, including program code, through thenetwork(s), network link 240, and communication interface 238. Althoughone network link 240 is shown, in alternative embodiments, communicationinterface 238 can provide coupling to a plurality of network links, eachof which connected to one or more local networks. In some embodiments,computer system 220 may receive data from one network, and transmit thedata to another network. Computer system 220 may process and/or modifythe data before transmitting it to another network.

Although particular embodiments have been shown and described, it shouldbe understood that the above discussion is intended to be illustrativeand not limiting, and various changes and modifications may be madewithout departing from the scope of embodiments or of the invention. Forexample, persons skilled in the art will appreciate that various typesof regularly dividing cells, including repair cells, can be identified,distinguished from pre-malignant or malignant cells, and rejected toprevent the normal cells from being selected for further review andanalysis. Further, persons skilled in the art will appreciate thatembodiments can be applied to portions of or all of an image.Additionally, different numbers of wavelengths can be utilized asnecessary. Moreover, persons skilled in the art will appreciate thatalthough the flow charts and description show and describe a methodinvolving a certain order of steps, steps can be performed in differentorders to achieve the same result.

What is claimed is:
 1. A method of processing an image of a biologicalspecimen using an image processor, the specimen image including aplurality of cells, the method comprising: identifying a component ofone of the cells in the image; and evaluating a characteristic of thecomponent in order to determine whether the respective cell having thecomponent has a likelihood of being a cancerous cell or a repair cell,wherein evaluating a characteristic of the component comprises:comparing one or more characteristics of the component to one or morecharacteristics of a component of a cell known to be cancerous;comparing one or more characteristics of the component to one or morecharacteristics of a component of a cell known to be a repair cell; anddetermining, based on the respective comparisons, whether the respectivecell having the component has a likelihood of being a cancerous cell ora repair cell.
 2. The method of claim 1, wherein the characteristiccomprises a physical characteristic.
 3. The method of claim 1, whereinthe characteristic comprises a statistical characteristic.
 4. The methodof claim 1, wherein the identified component comprises a nucleoli. 5.The method of claim 1, wherein identifying a component comprises using amulti-wavelength spectral analysis to identify the component.
 6. Themethod of claim 1, wherein evaluating a characteristic of the componentfurther comprises using multi-wavelength spectral analysis to determinetransmission characteristics of the component.
 7. The method of claim 1,further comprising: determining that the cell having the component has alikelihood of being cancerous; and presenting the cell to a reviewer forfurther review.
 8. A method of processing an image of a biologicalspecimen using an image processor, the specimen image including aplurality of cells, the method comprising: identifying a component ofone of the cells in the image; and evaluating a characteristic of thecomponent in order to determine whether the respective cell having thecomponent has a likelihood of being a cancerous cell or a repair cell,wherein the identified component comprises a nucleus, and whereinevaluating a characteristic of the component comprises evaluatingtransmission characteristics of the nucleus, and wherein evaluatingtransmission characteristics of the nucleus comprises comparingtransmission characteristics of the nucleus to transmissioncharacteristics of nuclei of cells known to be cancerous, comparingtransmission characteristics of the nucleus to transmissioncharacteristics of nuclei of cells known to be repair cells, anddetermining, based on the respective comparisons, whether the respectivecell having the identified nucleus has a likelihood of being a cancerouscell or a repair cell.
 9. A method of processing an image of abiological specimen using an image processor, the specimen imageincluding a plurality of cells, the method comprising: identifying acomponent of one of the cells in the image; and evaluating acharacteristic of the component in order to determine whether therespective cell having the component has a likelihood of being acancerous cell or a repair cell, wherein evaluating a characteristic ofthe component comprises: determining a repair score for the component;determining a cancer score for the component; comparing the repair scoreto the cancer score; and determining, based on the comparison, whetherthe respective cell having the component has a likelihood of being acancerous cell or a repair cell.
 10. The method of claim 9, wherein theidentified component comprises a nucleoli or nucleus.